Ironing out some kinks when running evals with high parallelism #2280
Conversation
|
dosubot
bot
added
the
size:L
|
Hello @cte, Thank you for your contribution! After reviewing the pull request, it seems that the changes span across multiple areas, including task processing improvements, form submission logic updates, schema enhancements, and database configuration changes. To ensure clarity and ease of review, it might be beneficial to split this pull request into smaller, more focused ones. Here are some suggestions on how the changes could be grouped:
By splitting the pull request, it will be easier for reviewers to focus on specific areas and provide more targeted feedback. Let me know if you have any questions or need further assistance. Best regards, |
![ellipsis-dev[bot]](https://avatars.githubusercontent.com/in/64358?s=60&v=4){kind=small}
| console.log(args) | ||
| } | ||
|
|
||
| this.logfile = path.join(getWorkspacePath(), "roo-code-messages.log") |
There was a problem hiding this comment.
Consider using a dedicated and consistent logging directory instead of getWorkspacePath() for the logfile. Using the workspace path may lead to inconsistent log locations when the extension is used in different projects. It might be better to use the extension's global storage (e.g. from context) to store logs.
| this.logfile = path.join(getWorkspacePath(), "roo-code-messages.log") | |
| this.logfile = path.join(this.context.globalStorageUri.fsPath, "roo-code-messages.log") |
dosubot
bot
added
size:XL
Context
Implementation
Screenshots
How to Test
Get in Touch
Important
Enhance evals with increased concurrency, improved task handling, web app updates, and enable WAL mode for SQLite.
MAX_CONCURRENCYto 20 andTASK_TIMEOUTto 10 minutes inindex.ts.javascripttest command timeout to 15s inindex.ts.TaskResultandTaskPromisetypes inindex.ts.processTaskResult()to manage task promises inindex.ts.runExercise()to handle task retries and disconnections inindex.ts.format-tokens.ts.Homecomponent to display token usage and add navigation button inhome.tsx.Runcomponent to conditionally showConnectionStatusinrun.tsx.db:enable-walscript inpackage.jsonandenable-wal.mtsto enable WAL mode for SQLite.db.tsto set connection concurrency to 50.layout.tsx.createRunSchemato userooCodeSettingsSchemainschemas.ts.This description was created by
for ad69957. It will automatically update as commits are pushed.